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MEMORANDUM  FOR  ASSISTANT  SECRETARY  OF  DEFENSE  (COMMAND, 

CONTROL,  COMMUNICATIONS,  AND  INTELLIGENCE) 

SUBJECT :  Audit  Report  on  Development  and  Implementation  of  Wide  Area 
Workflow-Receipts  and  Acceptance  (Report  No.  D-2002-018) 


We  are  providing  this  report  for  review  and  comment.  We  considered 
management  comments  on  a  draft  of  this  report  when  preparing  the  final  report. 

DoD  Directive  7650.3  requires  that  all  recommendations  be  resolved  promptly. 
Recommendations  1.,  2. a.,  2.d.,  and  2.f.  remain  unresolved.  Therefore,  we  request 
that  the  Chief  Information  Officer,  DoD,  Assistant  Secretary  of  Defense  for  Command, 
Control,  Communications,  and  Intelligence  respond  to  Recommendation  L,  and 
provide  additional  comments  to  Recommendations  2. a.,  2.d.,  and  2.f.  by 
January  28,  2002. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  Questions  on  the  audit 
should  be  directed  to  Ms.  Kathryn  M.  Truex  at  (703)  604-9139  (DSN  664-9139) 
(kmtruex@dodig.osd.mil)  or  Ms.  Jacqueline  L.  Wicecarver  at  (703)  604-9144 
(DSN  664-9144)  (jwicecarver@dodig.osd.mil).  See  Appendix  B  for  the  report 
distribution.  Audit  team  members  are  listed  inside  the  back  cover. 

David  K.  Steensma 
Acting  Assistant  Inspector  General 
for  Auditing 
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Development  and  Implementation  of  Wide  Area 
Workflow-Receipts  and  Acceptance 


Executive  Summary 


Introduction.  The  Under  Secretary  of  Defense  (Comptroller)  issued  Management 
Reform  Memorandum  No.  2  on  May  21,  1997,  which  directed  that  DoD  move  to  a 
totally  paper-free  contract  writing,  administration,  finance,  and  auditing  process  by 
January  1,  2000.  Although  nmnerous  actions  were  initiated,  DoD  has  not  yet  reached 
this  goal.  One  of  the  initiatives  was  the  development  of  the  Wide  Area 
Workflow-Receipts  and  Acceptance  (WAWF-RA)  application,  which  was  to  provide 
paperless  contracting,  receiving,  and  invoicing  DoD-wide.  The  WAWF-RA  allows 
vendors  to  submit  their  invoices  and  receiving  reports  electronically  to  DoD  and  route 
them  through  a  workflow  system  for  inspection,  acceptance,  receiving,  and  payment. 

Objectives.  The  overall  objective  of  the  audit  was  to  evaluate  the  adequacy  of  the 
development  and  implementation  of  the  WAWF-RA  application.  An  additional 
objective  was  to  determine  whether  the  WAWF-RA  application  was  compliant  with 
national  and  DoD  security  requirements.  Because  of  the  status  of  the  WAWF-RA 
development  and  implementation,  we  did  not  review  compliance  with  security 
requirements.  We  also  assessed  the  management  control  program  as  it  relates  to  the 
objective  of  the  audit. 

Results.  After  3  years  and  $8.4  million  expended,  the  Defense  Electronic  Business 
Program  Office  (DEBPO),  formerly  called  the  Joint  Electronic  Commerce  Program 
Office,  initiative  of  the  WAWF-RA  has  realized  limited  acceptance  and  use.  As  a 
result,  WAWF-RA  has  not  been  effective  in  providing  a  DoD-wide  solution  for  receipts 
and  acceptance  in  the  end-to-end  procurement  process.  Also,  WAWF-RA  has  not  yet 
achieved  the  goals  of  the  FY  2001  Authorization  Aet  to  implement  a  means  for 
electronic  claims  for  payment  processing  by  October  2002.  See  the  Finding  section  of 
the  report  for  details  on  the  audit  and  Appendix  A  for  details  of  the  review  on  the 
management  control  program. 

Summary  of  Reconuneudations.  We  recommend  the  DoD  Chief  Information  Officer 
determine  whether  WAWF-RA  will  perform  the  funetions  the  Military  Departments 
require  by  October  1,  2002.  Also,  the  DoD  Chief  Information  Officer,  along  with  the 
Chief  Information  Officers  of  the  Army,  Navy,  and  Air  Force,  should  require  the 
Military  Departments  to  adopt  and  implement  the  WAWF-RA  if  determined  that  the 
application  will  satisfy  DoD  needs  for  receipts  and  acceptance.  In  addition,  we 
recommend  the  DoD  Chief  Information  Officer  direct  the  DEBPO  to  determine  and 


document  whether  the  material  inspection  and  receiving  report  was  re-engineered.  The 
DEBPO  must  also  develop  an  acquisition  strategy;  prepare  a  memorandum  of 
agreement  with  DoD  Components;  perform  and  document  an  analysis  of  alternatives 
and  an  economic  analysis;  and  develop  an  independent  Management  Control  Program. 

Management  Comments.  The  Director,  Electronic  Business  and  Knowledge 
M^agement  did  not  address  whether  WAWF-RA  would  perform  the  functions  the 
Military  Departments  require  by  October  1,  2002.  The  Director,  Electronic  Business 
and  Knowledge  Management  concurred  with  all  other  recommendations.  Specifically, 
a  Joint  Requirements  Board  for  WAWF-RA  allows  for  an  ongoing  re-engineering  of  the 
material  inspection  and  receiving  report  process  as  new  versions  of  the  software  are 
released.  DEBPO  will  formally  document  the  WAWF-RA  Acquisition  Plan  by 
December  31,  2001,  and  complete  the  memorandums  of  agreement  with  Military 
Departments  and  Defense  agencies  by  June  30,  2002.  DEBPO  will  complete  an 
analysis  of  alternatives  as  future  versions  of  WAWF-RA  are  developed  and  will 
conduct  an  economic  analysis  to  be  used  as  a  baseline  in  evaluating  technologies  for 
future  releases  by  June  30,  2002.  Finally,  DEBPO  has  put  in  place  a  configuration 
management  system  and  a  monthly  structured  project  review  process  has  been  refined 
to  carefully  scrutinize  WAWF-RA  cost,  schedule,  and  performance.  A  discussion  of 
the  management  comments  is  in  the  Finding  section  of  the  report,  and  the  complete  text 
is  in  the  Management  Comments  section. 

Audit  Response.  The  Director,  Electronic  Business  and  Knowledge  Management  did 
not  address  or  determine  whether  WAWF-RA  would  perform  the  Actions  the  Military 
Departments  require.  Further,  the  Director’s  comments  were  not  fully  responsive  on 
recommendations  that  involved  determining  whether  the  material  inspection  and 
receiving  report  process  was  re-engineered,  performing  an  analysis  of  alternatives,  or 
developing  an  independent  Management  Control  program.  Therefore,  we  request  that 
the  DoD  Chief  Information  Officer  provide  additional  comments  by  January  28,  2002. 


Table  of  Contents 


Executive  Summary 


Introduction 

Background 

Objectives 


Finding 

Development  and  Implementation  of  Wide  Area  Workflow-Receipts  and 
Acceptance 

Appendixes 

A.  Audit  Process 

Scope 

Methodology 

Management  Control  Program  Review 
Prior  Coverage 

B.  Report  Distribution 


5 


17 

18 
18 

19 

20 


Management  Comments 

Assistant  Secretary  of  Defense  for  Command,  Control,  Communications, 
and  Intelligence 


Background 


The  Under  Secretary  of  Defense  (Comptroller)  issued  Management  Reform 
Memorandum  No.  2  on  May  21,  1997,  which  directed  that  DoD  move  to  a 
totally  paper-free  contract  writing,  administration,  finance,  and  auditing  process 
by  January  1,  2000.  Although  numerous  actions  were  initiated,  DoD  has  not 
yet  achieved  this  goal.  One  of  the  initiatives  was  the  development  of  the  Wide 
Area  Workflow-Receipts  and  Acceptance  (WAWF-RA)  application,  which  was 
to  provide  paperless  contracting,  receiving,  and  invoicing  DoD-wide.  The 
WAWF-RA  allows  vendors  to  submit  their  invoices  and  receiving  reports 
electronically  to  DoD  and  route  them  through  a  workflow  system  for  inspection, 
acceptance,  receiving,  and  payment. 

DoD  Fiscal  Year  2001  Authorization  Act.  Section  1008  of  the  DoD  Fiscal 
Year  2001  Authorization  Act  entitled  “Electronic  Submission  and  Processing  of 
Claims  for  Contract  Payments,”  states  that  the  Secretary  of  Defense  shall 
require  that  any  claim  for  payment  under  a  DoD  contract  be  submitted  to  the 
DoD  in  electronic  form.  The  Act  also  requires  the  Secretary  of  Defense  to 
provide  policies,  requirements,  and  procedures  for  using  electronic  means  for 
the  submission  of  claims  for  payment  to  DoD.  Implementation  is  to  occur  no 
later  than  October  1,  2002.  The  Secretary  of  Defense  was  required  to  submit, 
by  March  30,  2001,  a  plan  for  implementation  of  the  requirements,  to  the 
Committee  on  Armed  Services  of  the  Senate  and  House  of  Representatives. 
However,  the  report  was  not  submitted  to  Congress  until  July  6,  2001 .  The 
Under  Secretary  of  Defense  (Comptroller)  did  not  concur  with  the  report 
because  the  office  believed  DoD  should  have  been  able  to  meet  a  June  30,  2001, 
deadline  for  implementing  electronic  invoicing.  The  Under  Secretary  of 
Defense  (Comptroller)  believed  that  current  initiatives  could  have  been 
accelerated  to  ensure  that  electronic  invoicing  was  fully  implemented  throughout 
DoD  by  October  1,  2001. 

Defense  Electronic  Business  Program  Office.  The  Deputy  Secretary  of 
Defense  designated  the  Defense  Electronic  Business  Program  Office  (DEBPO), 
formerly  called  the  Joint  Electronic  Commerce  Program  Office,  as  the  DoD 
Executive  Agent  on  May  20,  1998.  DEBPO  is  responsible  for  promoting  and 
coordinating  DoDs  implementation  planning,  execution,  and  integration  of 
common  Electronic  Business  (EB)/Electronic  Commerce  (EC)  services. 

The  Chief  Information  Officer  (CIO),  DoD,  is  positioned  under  the  Assistant 
Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence). 
The  DoD  CIO  is  responsible  for  providing  direction  and  oversight  to  DEBPO. 
The  Defense  Information  Systems  Agency  and  the  Defense  Logistics  Agency  are 
required  to  provide  all  necessary  personnel,  facilities,  funding,  administration, 
and  logistical  support  to  DEBPO. 

DoD  Electronic  Business/Electronic  Commerce  Strategic  Pl£m.  The  Deputy 
Secretary  of  Defense  made  the  DoD  CIO  responsible  for  coordination  of  all  EB 
and  EC  initiatives.  The  DoD  CIO  formulated  a  DoD  team  to  set  up  the  DoD 
Electronic  Business/Electronic  Commerce  Strategic  Plan  (Strategic  Plan)  in 
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May  1999.  The  Strategic  Plan  sets  forth  the  DoD  EB/EC  vision  and  establishes 
goals,  objectives,  and  associated  strategies.  The  Strategic  Plan  is  intended  to 
permit  DoD  to  take  advantage  of  EB/EC  best  practices  and  initiatives  occurring 
within  and  among  the  Federal  agencies  and  their  business  parmers. 

Material  Inspection  and  Receiving  Report  Paper  Process.  The  DD 
Form  250,  Material  Inspection  and  Receiving  Report  is  the  key  multi-purpose 
business  form  that  supports  contracting  functions.  The  materid  inspection  and 
receiving  report  is  used  by  defense  industry  as  a  commercial  invoice  for 
payment  and  by  DoD  to  inspect  and  accept  goods  and  services,  as  well  as 
notification  of  shipment. 

The  material  inspection  and  receiving  report  paper  process  requires  numerous 
paper  copies  to  be  printed,  distributed,  and  exchanged  by  DoD  and  the 
contractor.  The  same  documents  are  retained  in  storage  at  multiple  locations 
and  duplicate  audit  functions  are  performed  at  various  agencies.  The 
reconciliation  process  of  matching  the  receipt,  invoice  and  contract  often  results 
in  document  rejection,  which  can  delay  payments  and  require  rework. 

General  Accounting  Office  Report.  Although  the  DoD  CIO  is  responsible  for 
DEBPO,  the  General  Accounting  Office  noted  in  its  report,  “Electronic 
Commerce  Implementation  Strategy  Can  be  Improved,”  July  2000,  that  the 
strucmre  of  DEBPO  has  created  problems  for  the  DoD  CIO.  The  report  states 
that  the  structure  diluted  the  DoD  CIOs  authority  over  DEBPO  and  has  caused 
the  DoD  CIO  to  be  excluded  from  the  decision-making  process  and  the  chain  of 
command.  The  report  recommended  that  DoD  ensure  that  the  EC  program 
office  has  clear  lines  of  authority  and  funding  necessary  to  implement  a 
DoD-wide  Program.  The  report  recommended  the  Secretary  of  Defense  direct 
the  DoD  CIO  to  develop  an  implementation  plan  that  explicitly  addressed  the 
DoD-wide  goals,  objectives,  and  strategies  of  the  Strategic  Plan.  The  Director 
for  Defense  Reform  concurred  with  the  recommendation  and  stated  that  the 
Strategic  Plan  would  be  updated  to  reflect  forthcoming  policies  on  electronic 
business.  However,  as  of  November  2001,  the  Strategic  Plan  had  not  been 
updated. 

WAWF-RA  Objectives.  DEBPO  created  the  WAWF-RA’  application  as  a 
prototype  to  eliminate  paper  from  the  receipts  and  acceptance,  invoice,  and 
payments  process  of  the  DoD  contracting  life-cycle.  The  WAWF-RA  contains 
the  documents  required  in  the  material  inspection  and  receiving  report  process  in 
a  virtual,  or  paperless,  folder  that  is  viewed  on  the  web.  The  objectives  of 
WAWF-RA  were  to  create  an  EC  environment  to  improve  bill  paying  and 
accounting  operations,  reduce  operating  costs,  and  reduce  unmatched 
disbursements.  The  WAWF-RA  would  accomplish  those  objectives  by 
eliminating  paper-based  support  functions,  increasing  document  accuracy,  and 
providing  secure  and  auditable  transactions. 


’A  second  application  of  Wide  Area  Workflow  involves  contract  closeout.  This  functionality  focuses  on 
the  closing  out  of  contracts  and  the  electronic  routing  and  distribution  of  the  paperwork  necessary  to 
accomplish  this.  The  DEBPO  began  the  application  definition  in  late  Fiscal  Year  2000  but  Wide  Area 
Workflow-Contract  Closeout  has  not  yet  become  operational. 
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Wide  Area  Workflow-Receipts  and  Acceptance  Process.  The  following 
figure  illustrates  the  WAWF-RA  process  and  the  flow  of  data  within 
WAWF-RA. 


Overview  of  the  Wide  Area  Workflow 
Receipts  and  Acceptance  Process 


Contracting 

Officer 


Source:  Defense  Electronic  Business  Program  Office 


Wide  Area  Workflow-Receipts  and  Acceptance  Process 

Once  a  contracting  officer  creates  the  contract,  it  can  be  digitized  and  stored  in 
the  Electronic  Document  Access  and  be  made  available  to  the  contracting 
officer,  vendor.  Government  acceptor/inspector,  and  the  payment  official 
through  a  virtual  folder.  When  a  contract  calls  for  goods  to  be  shipped,  the 
vendor  creates  an  invoice  and  a  material  inspection  and  receiving  report  on-line 
using  the  WAWF-RA  interactive  web  form.  The  vendor  enters  the  contract 
information,  such  as  the  name,  address,  and  contract  number.  The  invoice  and 
material  inspection  and  receiving  report  are  stored  in  the  virtual  folder  in 
WAWF-RA.  The  application  generates  an  e-mail  message  to  notify  the 
Government  official  that  he  or  she  needs  to  review  and  process  the  docimients. 
The  inspector/acceptor  receives,  validates,  and  digitally  si^s  the  material 
inspection  and  receiving  report.  The  WAWF-RA  application  then  generates  an 
e-mail  message  to  notify  the  payment  official  that  there  are  documents  ready  to 
be  processed.  The  paying  official  compares  the  contract,  invoice,  and  material 
inspection  and  receiving  report  in  the  virtual  folder  to  verify  that  all  the 
information  is  correct  on  all  the  documents.  The  vendor  payment  is  then 
electronically  deposited  in  the  vendor’s  bank  through  electronic  funds  transfer. 
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Objectives 


The  overall  objective  of  the  audit  was  to  evaluate  the  adequacy  of  the 
development  and  implementation  of  the  WAWF-RA  application.  An  additional 
objective  was  to  determine  whether  the  WAWF-RA  application  was  compliant 
with  national  and  DoD  security  requirements.  Because  of  the  status  of  the 
WAWF-RA  development  and  implementation,  we  did  not  review  compliance 
with  security  requirements.  We  also  assessed  the  management  control  program 
as  it  relates  to  the  objective  of  the  audit.  See  Appendix  A  for  a  discussion  of  the 
audit  scope  and  methodology,  management  controls,  and  prior  coverage. 
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Development  and  Implementation  of 
Wide  Area  Workflow-Receipts  and 
Acceptance 

After  3  years  and  $8.4  million  expended,  the  DEBPO  initiative  of 
WAWF-RA  has  realized  limited  acceptance  and  use.  Acceptance  and 
use  of  WAWF-RA  has  been  limited  because  DoD  has  not  effectively 
managed  the  development  and  implementation  of  WAWF-RA. 
Specifically,  DEBPO  did  not  develop  the  WAWF-RA  consistent  with 
information  technology  and  program  acquisition  guidelines.  DEBPO  did 
not; 

•  analyze  and  revise  the  receipts  and  acceptance  business  process 
before  it  began  developing  ftie  WAWF-RA, 

•  develop  a  WAWF-RA  acquisition  strategy, 

•  prepare  a  memorandum  of  agreement  with  DoD  Components, 

•  conduct  acquisition  decision  analysis,  and 

•  record,  preserve,  and  make  available  sufficient  information  to 
substantiate  the  acquisition  and  development  of  WAWF-RA. 

In  addition,  the  Military  Departments  have  not  adopted  WAWF-RA.  As 
a  result,  WAWF-RA  has  not  been  effective  in  providing  a  DoD-wide 
solution  for  receipts  and  acceptance  in  the  end-to-end  procurement 
process.  Also,  WAWF-RA  has  not  been  effective  in  meeting  the  goals 
of  the  FY  2001  Authorization  Act  to  implement  a  means  for  electronic 
claims  processing  by  October  2002. 

Policy  and  Directives 

Defense  Reform  Initiative  Directives.  The  Deputy  Secretary  of  Defense  issued 
Defense  Reform  Initiative  Directives  (DRIDs)  to  address  Management  Reform 
Memorandum  No.  2.  DRID  Nos.  33  and  43  addressed  the  material  inspection 
and  receiving  report  process  and  the  establishment  of  DEBPO. 

Defense  Reform  Initiative  Directive  No.  33.  The  Deputy  Secretary  of 
Defense  published  the  Department  of  Defense  Reform  Initiative  Directive 
(DRID)  No.  33,  “Paperless  DD  Form  250,  Material  Inspection  and  Receiving 
Report,”  on  April  13,  1998.  DRID  No.  33  identified  the  material  inspection 
and  receiving  report  process,  as  having  great  potential  to  be  re-engineered  and 
transitioned  to  a  paperless  environment.  The  Deputy  Secretary  of  Defense 
tasked  the  Defense  Contract  Management  Agency  to  lead  an  effort  to 
re-engineer  the  process  with  the  aim  of  making  it  paperless. 


5 


Defense  Reform  Initiative  Directive  No.  43.  The  Deputy  Secretary  of 
Defense  directed  the  establishment  of  DEBPO  in  DRID  No.  43,  “Defense-wide 
Electronic  Commerce,”  on  May  20,  1998.  DEBPO  was  to  allow  DoD  to 
centralize  EC  policy  recommendations,  planning,  and  coordination  to  ensure 
consistent  implementation.  DRID  No.  43  required  DEBPO  to  develop  a  charter 
and  coordinate  it  with  DoD  officials  including  the  Under  Secretary  of  Defense 
(Acquisition,  Technology,  and  Logistics),  Under  Secretary  of  Defense 
(Comptroller),  DoD  CIO,  and  Secretaries  of  the  Military  Departments. 

Further,  the  charter  was  to  be  reflected  in  the  DoD  Directives  system. 

Information  Technolo^  and  Program  Acquisition  Guidelines.  Information 
technology  and  acquisition  guidelines  exist  for  the  development  of  automated 
information  systems.  The  Clinger-Cohen  Act  of  1996;  Office  of  Management 
and  Budget  Circular  A-130,  “Management  of  Federal  Information  Resources,” 
February  8,  1996;  and  DoD  5000.2-R,  “Mandatoiy  Procedures  for  Major 
Defense  Acquisition  Programs  (MDAPS)  and  Major  Automated  Information 
System  (MAIS)  Acquisition  Programs,”  October  23,  2000,  should  be  a^ered  to 
when  acquiring  or  developing  new  information  technology.  The  guidelines 
require  that  the  acquisition  for  information  technology  be  delivered  at  acceptable 
costs  and  within  reasonable  time  frames.  Further,  the  guidelines  require  that  the 
acquisition  contribute  to  improvements  in  mission  performmce.  The  acquisition 
is  to  support  a  process  that  has  been  completely  analyzed,  simplified,  and 
redesigned. 

Clinger-Cohen  Act  of  1996.  The  Clinger-Cohen  Act  of  1996  requires 
DoD  to  analyze  its  mission  and  revise  its  processes  before  significantly  investing 
in  information  technology.  The  information  technology  should  be  delivered 
within  18  months  after  DoD  issues  a  contract  solicitation.  The  Secretary  of 
Defense  reiterated  the  goals  and  objectives  of  the  Clinger-Cohen  Act  in  a 
memorandum  in  June  1997.  The  keys  to  a  successful  implementation  of 
Information  Technology  Management  Reform  included  speedy  implementation, 
unity  of  authority,  and  management  commitment. 

Office  of  Management  and  Budget  Circular  A-130.  Office  of 
Management  and  Budget  (0MB)  Circular  A-130,  requires  agencies  to  record, 
preserve,  and  make  accessible  sufficient  information  to  ensure  the  management 
and  accountability  of  agency  programs.  Federal  agencies  are  required  to 
incorporate  a  records  management  function  into  the  design,  development,  and 
implementation  of  information  systems. 

DoD  5000.2-R.  DoD  5000.2-R  provides  guidelines  for  acquisition  for 
information  technology.  DoD  5000.2-R  requires  program  managers  to  prepare 
an  analysis  of  alternatives.  Further,  each  program  manager  should  develop  and 
document  an  acquisition  strategy  to  serve  as  a  roadmap  for  program  execution 
from  program  initiation  through  post-production  support. 
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WAWF-RA  Acceptance  and  Use 


After  3  years  and  $8.4  million  expended,  the  DEBPO  initiative  of  WAWF-RA 
has  realized  limited  acceptance  and  use.  The  Deputy  Secretary  of  Defense 
established  DEBPO  under  DRID  No.  43  in  May  1998  to  promote  and 
coordinate  EC  throughout  DoD.  In  an  effort  to  promote  EC,  DEBPO  began 
developing  a  series  of  initiatives  to  automate  portions  of  the  DoD  procurement 
process.  One  of  those  initiatives  was  the  WAWF-RA.  DEBPO  initiated  the 
WAWF-RA  development  in  1998  to  automate  the  receipts  and  acceptance 
portion  of  the  end-to-end  procurement  cycle.  The  DEBPO,  at  the  direction  of 
the  Deputy  Secretary  of  Defense,  initiated  the  WAWF-RA  in  order  to  fulfill  the 
requirements  of  DRID  No.  33. 

WAWF-RA  Useage.  DoD  is  using  WAWF-RA  for  only  a  miniscule  portion  of 
contracting  actions.  DoD  purchased  $156  billion  in  goods  and  services  in 
15  million  purchasing  actions  during  FY  2000.  The  DEBPO  intended  all  of 
DoD  to  use  WAWF-RA.  However,  as  of  March  1,  2001,  WAWF-RA  had  only 
33  active  vendors  that  had  processed  a  total  of  $50  million  in  goods  and 
services. 

WAWF-RA  Status.  As  of  July  2001,  WAWF  was  in  pilot^  deplo5mient. 
According  to  DEBPO,  the  current  version  of  WAWF-!^  is  Version  1.3B  and  is 
considered  to  be  in  a  pilot  phase.  The  next  major  release  will  be  Version  2.0, 
which  was  to  be  fielded  in  three  increments  in  July  2001,  October  2001,  and 
January  2002. 

The  Military  Departments  have  three  pilot  sites  that  are  using  WAWF-RA  in  a 
limited  capacity.  The  Army  has  one  pilot  site  for  WAWF-]^  that  has  been  in 
operation  since  September  2000  which  processes  one  transaction  per  month. 

The  Navy  has  no  WAWF-RA  pilot  sites.  The  Air  Force  currently  has  two  pilot 
sites  using  WAWF-RA. 

Effectiveness  of  WAWF-RA  Development  and  Implementation 


Acceptance  and  use  of  WAWF-RA  has  been  limited  because  DoD  has  not 
effectively  managed  the  development  and  implementation  of  WAWF-RA. 
Specifically: 

•  DEBPO  did  not  develop  WAWF-RA  consistent  with  information 
technology  and  program  acquisition  guidelines,  and 

•  Military  Departments  have  not  adopted  the  WAWF-RA. 


test  location,  normally  used  to  see  if  an  application  operates  satisfactorily  before  placing  it  at  other 
locations. 
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Development  of  WAWF-RA.  DEBPO  did  not  develop  the  WAWF-RA 
application  consistent  with  information  technology  and  program  acquisition 
guidelines.  DEBPO  believed  that  because  this  was  a  protot5^e  acquisition,  they 
did  not  have  to  adhere  to  Clinger-Cohen  Act,  DoD  5000  Series  Acquisition 
Directives  and  Regulations,  and  0MB  Regulations. 

Specifically,  DEBPO  did  not: 

•  analyze  and  revise  the  receipts  and  acceptance  business  process 
before  it  began  developing  the  WAWF-RA, 

•  develop  a  WAWF-RA  acquisition  strategy, 

•  prepare  a  memorandum  of  agreement  with  DoD  Components, 

•  conduct  acquisition  decision  analysis,  and 

•  record,  preserve,  and  make  available  sufficient  information  to 
substantiate  the  acquisition  and  development  of  WAWF-RA. 

Business  Process  Analysis  and  Re-engineering.  The  Clinger-Cohen 
Act  of  1996  and  DoD  5000. 2-R  require  DoD  to  analyze  and  revise  processes 
before  making  significant  investments  in  information  technology.  DEBPO  did 
not  analyze  and  revise  the  receipts  and  acceptance  business  process  before 
developing  the  WAWF-RA.  The  Deputy  Secretary  of  Defense  stated,  in 
DRID  No.  33,  the  material  inspection  and  receiving  report  process  had  great 
potential  to  be  re-engineered  and  transitioned  to  a  paperless  environment.  On 
April  20,  1998,  the  Deputy  Secretary  of  Defense  appointed  a  working  group  to 
conduct  a  comprehensive  review  and  develop  recommendations  to  streamline  the 
material  inspection  and  receiving  report  process  with  the  aim  of  making  it 
paperless.  The  group  issued  the  results  of  its  review  in  a  report  on  the  material 
inspection  and  receiving  report  in  April  1999.  However,  DEBPO  began 
developing  the  WAWF-RA  application  at  a  cost  of  approximately  $2  million,  on 
July  30,  1998,  about  one  year  before  the  group  issued  its  results.  DEBPO 
stated  that,  at  inception,  the  WAWF-RA  was  developed  at  the  direction  and 
guidance  of  the  Deputy  Secretary  of  Defense  with  the  intent  on  providing  a 
paperless  solution  by  January  1,  2000.  DEBPO  did  not  re-engineer  the  material 
inspection  and  receiving  report  process. 

WAWF-RA  Acquisition  Strate^.  DoD  5000. 2-R  states  that  an 
acquisition  strategy  should  serve  as  a  guide  for  program  execution  from 
initiation  through  implementation.  An  acquisition  strategy  should  provide 
quantifiable  requirements,  time  frames  and  baselines,  and  measures  of 
performance  to  minimize  the  time  and  cost  of  completion.  DEBPO  documented 
that  it  was  responsible  for  implementing  EC  for  the  DoD.  DEBPOs  strategy  for 
doing  so  was  to  develop  prototype  applications,  such  as  WAWF-RA,  and  to 
adopt  selected  prototypes  and  technologies  for  DoD-wide  implementation. 
However,  DEBPO  did  not  outline  the  details  of  how  it  would  realistically 
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achieve  this  objective.  DEBPO  did  not  develop  a  WAWF-RA  acquisition 
strategy  or  document  and  develop  quantifiable  requirements,  time  frames  and 
baselines,  or  measures  of  performance. 

Memorandum  of  Agreement.  DoD  5000.2-R  states  that  a 
memorandum  of  agreement  (MO A)  should  be  developed  for  a  joint  program  to 
specify  the  relationship  and  responsibilities  of  the  program  manager.  Military 
Departments,  and  Defense  agencies.  DEBPO  did  not  develop  an  MO  A. 

A  detailed  MOA  could  have  provided  a  framework  for  a  cohesive  management 
structure.  Further,  it  would  have  served  as  a  vehicle  for  management 
commitment  and  participation  by  the  Military  Departments.  At  a  minimum,  the 
MOA  should  address  system  requirements,  fonding,  manpower,  and  the 
approval  process  of  the  operational  requirements  document  and  other  program 
documentation. 

Because  DEBPO  did  not  develop  an  MOA,  the  WAWF-RA  development  lacked 
a  cohesive  management  structure  and  authority  and  WAWF-RA  ownership  is 
not  apparent.  For  example,  the  DoD  CIO  is  responsible  for  oversight  of  the 
DEBPO,  the  WAWF-RA  Program  Manager.  The  Defense  Information  Systems 
Agency  provides  funding,  while  Defense  Logistics  Agency  provides  resources. 
Therefore,  the  two  organizations  could,  in  effect,  have  more  control  over  the 
development  than  the  DoD  CIO  or  the  business  process  owners  to  include  the 
Military  Departments.  Further,  the  DoD  Strategic  Plan  lists  the  Defense 
Contract  Management  Agency  as  the  official  responsible  for  the  material 
inspection  and  receiving  report.  However,  DEBPO  has  been  responsible  for  the 
development  of  the  WAWF-RA.  The  WAWF-RA  requirements  document  does 
not  identify  Military  Departments  as  a  WAWF-RA  partner.  Because  of  the  lack 
of  clear  lines  of  responsibility,  DEBPO  should  develop  an  MOA  with  the 
Military  Departments  and  those  Defense  agencies. 

Acquisition  Decision  Analysis.  DoD  5(XX).2-R  is  the  model  for 
acquiring  non-major  automated  information  systems.  Analysis  of  alternatives 
and  the  economic  analysis  should  be  the  basis  for  developing  or  acquiring  new 
information  technology.  DEBPO  did  not  prepare  either  for  WAWF-RA. 

Analysis  of  Alternatives.  DoD  5000.2-R  requires  program 
managers  to  develop  an  analysis  of  alternatives  to  aid  and  document  decision 
making  by  showing  the  risk,  uncertainty,  and  the  relative  advantages  and 
disadvantages  of  the  alternatives  being  considered.  DEBPO  did  not  prepare  an 
analysis  of  alternatives.  However,  DEBPO  acknowledged  that  the  April  1999 
report  on  the  material  inspection  and  receiving  report  identified  four  systems,  in 
several  stages  of  development,  that  made  the  process  paperless: 

•  Defense  Contract  Management  Command  Electronic  Data 
Interchange  DD  Form  250, 

•  Paperless  Contracting  Working  Integrated  Process  Team  Wide  Area 
Workflow, 
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•  Federal  Systems  Integration  and  Management  Center  Tracking  and 
Ordering  System,  and 

•  Defense  Medical  Logistics  Standard  Support. 

The  April  1999  report  stated  that  none  of  the  four  systems,  including 
WAWF-RA,  satisfied  the  requirements  of  all  DoD  users.  The  April  1999  report 
did  not  document  the  risk,  uncertainty,  advantages,  and  disadvantages  of  the 
alternatives  being  considered.  Regardless,  DEBPO  did  not  wait  for  the  results 
of  the  study  and  initiated  the  development  of  the  WAWF-RA. 

The  April  1999  report  stated  that  information  technology  has  an  18-month 
turnover  and  solutions  must  be  timely.  The  Clinger-Cohen  Act  of  1996  also 
states  that  information  technology  should  be  delivered  within  18  months  after 
solicitation.  Since  DEBPO  began  developing  the  WAWF-RA  more  than  3  years 
ago,  information  technology  has  changed  significantly.  As  such,  DEBPO 
should  reassess  the  WAWF-RA  to  determine  whether  it  is  the  best  solution  for 
receipts  and  acceptance.  DEBPO  should  develop  an  analysis  of  alternatives  to 
identify  the  advantages  and  disadvantages  of  current  technology  and  the  current 
version  of  WAWF-RA  to  validate  that  it  is  the  best  solution  before  DoD  incurs 
further  expenses. 

Economic  Analysis.  DoD  Instruction  7041.3,  “Economic 
Analysis  for  Decisionmaking,”  November  7,  1995,  requires  DoD  to  develop  an 
economic  analysis  that  includes  the  total  costs  and  benefits  of  each  feasible 
alternative  for  new  information  technology  investments.  DEBPO  did  not 
develop  an  economic  analysis.  Effective  use  of  an  economic  analysis,  in 
conjunction  with  an  analysis  of  alternatives,  gives  program  managers  a  valid 
basis  for  evaluating  the  feasibility  of  alternatives.  DEBPO  should  develop  an 
economic  analysis  that  includes  total  costs  and  benefits  of  each  feasible 
alternative  identified  in  the  analysis  of  alternatives. 

Program  Documentation.  0MB  Circular  A- 130  requires  Federal 
agencies  to  record,  preserve,  and  make  information  available.  Additionally, 
DoD  5000.2-R  states  that  program  managers  are  fully  responsible  and 
accountable  for  the  cost,  schedule,  and  performance  of  the  system  development. 
Program  documentation  is  the  means  for  accountability  of  program  compliance 
and  performance.  DEBPO  program  documentation  did  not  demonstrate  that 
WAWF-RA  was  developed  in  accordance  with  Federal  and  DoD  Regulations 
such  as  the  Clinger-Cohen  Act  of  1996  or  DoD  5000.2-R. 

DEBPO  is  the  DoD  executive  agent  for  EB/EC  and  is  responsible  for  the 
development  of  WAWF-RA.  The  DEBPO  personnel  stated  that  documentation 
did  not  exist  or  was  not  readily  available.  Personnel  stated  that  the  constant 
change  of  the  WAWF-RA  program  managers  (four  different  program  managers 
since  1998)  had  caused  those  documentation  problems.  Also,  the  existing 
documentation  was  not  centrally  located  within  the  DEBPO  program  office,  or 
was  lost. 
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Federal  and  DoD  guidelines  encourage  continuous  examination  and  adoption  of 
innovative  practices.  However,  the  Federal  and  DoD  guidelines  do  not  absolve 
program  managers  from  accountability  for  the  development  of  a  system.  The 
DEBPO,  as  the  program  manager,  should  have  developed  and  maintained 
WAWF-RA  documentation.  Specifically,  WAWF-RA  documentation  should 
have  demonstrated  that  WAWF-RA  was  developed  in  accordance  with  DoD 
Acquisition  Directives  and  Regulations;  the  Clinger-Cohen  Act;  and  0MB 
Circular  A-130. 

Prototyping  and  Standard  Application  Development.  The  DEBPO  did  not 
consider  the  DoD  5000  Series  Acquisition  Directives  and  Regulations 
applicable.  DEBPO  believed  that  it  did  not  have  to  comply  with  DoD 
acquisition  requirements  because  WAWF-RA  was  a  protore. 

A  prototyping  development  uses  a  trial  and  error  approach  to  define  and  refine 
the  requirements  after  the  initial  system  design.  Users  are  responsible  for 
specifying  the  requirements,  evaluating  the  iterations,  refining  requirements, 
and  communicating  changes.  In  addition: 

•  the  design  documentation  may  not  be  thorough,  making  fiimre 
maintenance  more  difficult,  and 

•  the  design  may  be  incomplete  and  result  in  a  highly  inefficient 
system. 

Prototyping  usually  places  the  application  into  operation  quicker  than  a  standard 
development,  but  has  associated  risks.  The  5000. 2-R  approach  to  information 
technology  development  defines  requirements  before  the  system  is  designed,  and 
has  enhanced  management  and  controls  over  the  development  process. 
Therefore,  the  associated  risks  are  lower.  Standard  development  does  not  give 
users  a  functional  system  as  quickly  as  a  prototyping  effort  because  application 
development  time  is  longer  than  that  of  prototyping. 

DEBPO  believed  DoD  acquisition  guidelines  were  not  applicable  because  it  was 
developing  a  prototype  and  because  developing  a  prototype  for  EC  was 
unchartered  territory.  DEBPO  should  have  considered  and  incorporated 
DoD  5000  guidelines  into  the  development  of  WAWF-RA  in  order  to  minimize 
the  risks  of  developing  an  application  that  is  insufficient. 

The  Military  Departments  Adoption  of  WAWF-RA.  The  Military 
Departments  have  not  adopted  WAWF-RA  as  the  DoD-wide  solution  to 
paperless  receipts  and  acceptance.  The  Army,  Navy,  and  Air  Force  have 
voiced  concerns  about  WAWF-RA  functionality.  According  to  the  Military 
Departments,  implementation  of  WAWF-RA  is  contingent  upon  the  following 
functions:  WAWF-RA  software  fimctionality  that  provides  for  the  automatic 
population  of  data  from  contract  documentation  for  invoice  and  acceptance 
documents;  sufficient  communications  infrastructure;  and  procedures  for  Public 
Key  Infrastructure  Certificate  issuance,  maintenance,  and  revocation. 

According  to  the  program  manager,  WAWF-RA  is  a  typical  system  that 
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continues  to  improve.  As  new  versions  of  WAWF-RA  become  available,  new 
features  and  phases  will  be  introduced  that  will  address  the  concerns  and 
requirements  of  the  Military  Departments. 

The  DoD  CIO  stated  that  WAWF-RA  is  the  target  system  for  processing 
invoices,  receipts,  and  acceptance  DoD-wide.  Therefore,  the  DoD  CIO  should 
determine  whether  WAWF-RA  will  perform  the  functions  the  Military 
Departments  require  by  October  1,  2002.  The  DoD  CIO,  along  with  the  CIOs 
of  the  Army,  Navy,  and  Air  Force,  should  require  the  Military  Departments  to 
adopt  and  implement  the  WAWF-RA  if  determined  that  the  application  will 
satisfy  DoDs  needs  for  receipts  and  acceptance. 

Impact  of  WAWF-RA  Initiative  Throughout  DoD 


WAWF-RA  has  not  been  effective  in  providing  a  DoD-wide  solution  for 
receipts  and  acceptance  in  the  end-to-end  procurement  process.  Also,  the 
WAWF-RA  has  not  been  effective  in  meeting  the  goals  of  the  FY  2001 
Authorization  Act  to  implement  a  means  for  electronic  claims  processing  by 
October  2002.  The  FY  2001  Authorization  Act  requires  that  DoD  contractors 
submit  electronic  claims  for  payment  by  October  1,  2002.  The  DoD  CIO  stated 
that  WAWF-RA  is  the  target  system  for  processing  invoices,  receipts,  and 
acceptance  DoD-wide.  The  Secretary  of  Defense  was  required  to  report  to 
Congress  by  March  30,  2001,  concerning  how  DoD  would  implement  electronic 
invoicing  by  October  1,  2002.  On  July  6,  2001,  the  report  was  signed  and  sent 
to  Congress.  The  report  states  that  DoD  has  developed  or  is  developing  various 
systems  to  enable  the  electronic  transmittal  of  receipt  and  acceptance 
information  and  invoice  information.  The  report  states  that  the  Military 
Departments  full  deployment  of  WAWF-RA  is  planned  to  begin  in  January  2002 
and  end  in  September  2002.  However,  the  Under  Secretary  of  Defense 
(Comptroller)  did  not  concur  with  the  report  because  the  DoD  Components 
would  not  confirm  the  implementation  dates  for  electronic  invoicing.  DoD  has 
not  reached  an  agreement  on  the  mechanism  and  implementation  date  for 
electronic  invoicing. 

DEBPO  has  taken  the  initiative  to  develop  WAWF-RA  as  a  means  to  make 
electronic  claims  for  payment  as  intended  in  the  FY2001  Authorization  language 
and  in  DRID  No.  33.  However,  the  Military  Departments  have  not  adopted 
WAWF-RA.  The  DoD  CIO  is  responsible  for  implementing  the  DoD-wide  EC 
initiatives  and  for  overseeing  DEBPO.  The  DoD  CIO  should  monitor  and 
approve  DEBPO  EC  initiatives  in  order  to  manage  the  risk,  uncertainty,  and  the 
relative  advantages  and  disadvantages  of  alternatives  for  receipts  and 
acceptance.  The  DoD  CIO  must  also  determine  if  the  Military  Departments  will 
be  capable  of  full  implementation  of  WAWF-RA  by  the  deadline  set  in  the 
FY  2001  Authorization  Act. 
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Recommendations,  Management  Comments,  and  Audit 
Response 


We  recommend  that  the  Chief  Information  Officer,  DoD,  Assistant 
Secretary  of  Defense  (Command,  Control,  Communications,  and 
Intelligence): 

1.  Determine  whether  Wide  Area  Workflow-Receipts  and  Acceptance 
will  perform  the  functions  that  the  Military  Departments  require  hy 
October  1, 2002,  and,  along  with  the  Chief  Information  Officers  of  the 
Army,  Navy,  and  Air  Force,  require  the  Military  Departments  to  adopt 
and  implement  the  Wide  Area  Workflow-Receipts  and  Acceptance  if  it 
is  determined  that  the  application  will  satisfy  DoDs  needs  for  receipts 
and  acceptance. 

Management  Comments.  The  DoD  CIO  did  not  comment  on  the 
recommendation.  We  request  that  the  DoD  CIO  provide  comments  in 
response  to  the  final  report. 

2.  Direct  the  Defense  Electronic  Business  Program  Office  to: 

a.  Determine  and  document  whether  the  material  inspection 
and  receiving  report  process  was  re-engineered  and  not  just  made 
paperless. 

Management  Comments.  The  Director,  Electronic  Business  and 
Knowledge  Management  concurred  and  stated  that  with  each  version  release 
of  WAWF-RA,  all  requirements  targeted  for  a  particular  release  are 
discussed  by  the  WAWF-RA  Joint  Requirements  Board,  allowing  for 
ongoing  re-engineering  of  the  process  as  new  requirements  and 
functionality  are  included  and  new  versions  of  the  software  released. 

Audit  Response.  Although  the  Director,  Electronic  Business  and 
Knowledge  Management  concurred,  the  comments  were  not  fully 
responsive.  The  Director  addressed  the  importance  of  the  Joint 
Requirements  Board  to  discuss  new  requirements  for  a  particular  release. 
However,  the  Director  does  not  address  the  intent  of  the  recommendation. 
DEBPO  must  determine  whether  WAWF-RA  has  been  re-engineered  to 
satisfy  DoDs  needs  for  receipts  and  acceptance.  If  that  is  not  the  case, 
DEBPO  must  accomplish  the  re-engineering  process  as  required  by  DRID 
No.  33.  We  request  that  the  DoD  CIO  reconsider  its  response  and  provide 
additional  comments  in  response  to  the  final  report. 
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b.  Develop  an  acquisition  strategy  that  defines  quantifiable 
requirements,  time  frames  and  baselines,  and  measures  of  performance 
for  the  paperless  receipts  and  acceptance  process. 

Management  Conunents.  The  Director,  Electronic  Business  and 
Knowledge  Management  concurred,  stating  that  there  is  no  formal 
WAWF-RA  acquisition  plan  or  strategy.  However,  a  process  has  been 
implemented  that  defines  quantifiable  requirements,  time  frames,  and 
baselines  through  the  auspices  of  the  Joint  Requirements  Board  for 
WAWF-RA.  Further,  measures  for  paperless  receipts  and  acceptance 
process  have  been  included  in  the  metrics  tracked  by  the  Paperless 
Contracting  Working  Integrated  Process  Team  and  continue  to  be  tracked  as 
that  effort  transferred  to  the  DoD  Cross-Functional  Implementation 
Integrated  Process  Team.  DEBPO  will  formally  document  the  WAWF-RA 
acquisition  plan  by  December  31,  2001. 

c.  Develop  a  memorandum  of  agreement  with  the  Military 
Departments  and  Defense  agencies  which  specifically  outlines  and 
details  the  relationship  and  responsibilities  for  the  Wide  Area 
Workflow-Receipts  and  Acceptance. 

Management  Comments.  The  Director,  Electronic  Business  and 
Knowledge  Management  concurred,  stating  responsibility  for  developing 
each  MOA  rests  with  DEBPO  until  WAWF-RA  is  turned  over  to  Executive 
Agents.  DEBPO  will  complete  development  of  each  MOA  in  coordination 
with  the  Executive  Agents  by  June  30,  2002. 

d.  Perform  and  document  an  analysis  of  alternatives  which 
identifies  the  advantages  and  disadvant^es  of  current  technology  and 
the  current  version  of  Wide  Area  Workflow-Receipts  and  Acceptance. 

Management  Comments.  The  Director,  Electronic  Business  and 
Knowledge  Management  partially  concurred  and  stated  that  initially  a 
proof  of  concept  examined  alternatives  through  the  demonstration  of  four 
independent  technology  solutions  in  1998,  followed  by  a  comprehensive 
search  of  commercial-off-the-shelf  products.  The  analysis,  conducted  in 
September  1998,  was  not  documented;  however,  it  led  to  the  technology 
selected,  which  was  a  proven  product  in  other  initiatives.  Subsequently, 
WAWF-RA  deployed  Version  2.0a,  adding  functionality  to  the  prototj^e 
version  1.3b.  A  new  analysis  at  this  time  comparing  WAWF-RA  to 
current  technologies  will  not  impact  the  acquisition  or  the  cost  to  the 
Government  in  any  significant  way,  as  the  effort  is  complete.  As  plans  for 
future  WAWF-RA  versions,  incorporating  additional  functionality,  are 
developed,  alternative  technologies  will  be  evaluated  and  documented 
through  the  WAWF-RA  Joint  Requirements  Board  as  the  Inspector 
General  has  recommended.  In  addition,  commercial-off-the-shelf  solutions 
for  workflow  functionality  are  frequently  being  evaluated  by  DEBPO. 
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Audit  Response.  Although  the  Director,  Electronic  Business  and 
Knowledge  Management  concurred,  the  comments  were  not  fully 
responsive.  According  to  contract  documentation,  in  July  1998,  a  contract 
was  awarded  to  develop  the  WAWF-RA  prototype  before  any  analysis  of 
alternatives  was  completed.  DEBPO  should  develop  a  current  analysis  of 
alternatives  to  identify  the  advantages  and  disadvantages  of  both  the 
current  technology  and  the  current  version  of  WAWF-RA.  This  analysis 
will  validate  that  WAWF-RA  remains  the  best  solution  before  DoD  incurs 
further  expenses  from  WAWF-RA.  We  request  that  the  DoD  CIO 
reconsider  its  response  and  provide  additional  comments  in  response  to  the 
final  report. 

e.  Perform  and  document  an  economic  analysis  that  includes 
the  total  costs  and  benefits  of  each  feasible  alternative. 

Management  Comments.  The  Director,  Electronic  Business  and 
Knowledge  Management  concurred,  stating  that  the  costs  of  the 
development  and  testing  of  the  current  version  are  sunk  costs  pd  an 
analysis  of  comparing  alternatives  will  not  impact  that  expenditure.  As 
new  requirements  are  reviewed,  the  process  now  includes  an  evaluation  of 
alternate  technologies  and  the  associated  cost  and  benefits  for  different 
approaches.  A  formal  economic  analysis  was  not  documented  for 
WAWF-RA.  Although  evaluating  the  costs  for  the  current  version  of 
WAWF-RA  to  other  technologies  at  this  time  would  seem  to  be  of 
somewhat  limited  value,  DEBPO  will  perform  and  document  an  economic 
analysis  to  be  used  as  a  baseline  in  evaluating  technologies  for  future 
releases.  This  economic  analysis  will  be  completed  by  June  30,  2002. 

f.  Develop  an  independent  Management  Control  Program  that 
provides  reasonable  assurance  with  clear  control  objectives  that  the 
development  of  new  information  technology  complies  with  the 
requirements  of  the  Clinger-Cohen  Act  of  1996  and  the  Office  of 
Management  and  Budget  Circular  A-130. 

Management  Comments.  The  Director,  Electronic  Business  and 
Knowledge  Management  concurred  and  stated  that  about  a  year  ago, 
DEBPO  put  in  place  a  configuration  management  system.  All  WAWF-RA 
contractor  documents  and  software  deliverables  are  properly  managed  and 
tracked  on  this  system.  The  Joint  Requirements  Board  defines  and 
prioritizes  functional  requirements,  recommends  system  enhancements, 
oversees  functional  and  operational  testing,  and  facilitates  cross-agency 
planning,  training  and  deplo5nnent.  To  further  improve  oversight  by  the 
project  manager,  a  monthly  structured  project  review  process  has  been 
refined  to  carefully  scrutinize  WAWF-RA  cost,  schedule,  and 
performance.  Taken  together,  these  initiatives  clearly  place  the 
WAWF-RA  leadership  in  a  position  to  assure  management  commitment 
and  oversight  leading  to  a  successful  implementation  of  WAWF-RA. 
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Audit  Response.  Although  the  Director,  Electronic  Business  and 
Knowledge  Management  concurred  with  the  recommendation,  the 
comments  were  not  fully  responsive.  DEBPO  has  put  controls  into  place 
within  its  acquisition  process;  however,  DEBPO  has  not  identified  a  plan 
to  test  these  controls.  The  response  does  not  address  a  plan  of  action  for 
developing  an  independent  management  control  program  with  clear  control 
objectives  that  reasonably  assures  compliance  with  the  Clinger-Cohen  Act 
and  0MB  Circular  A-130.  Also,  the  comments  do  not  address  a 
self-assessment  to  determine  if  the  control  objectives  are  achieving  the 
intended  results.  We  request  that  the  DoD  CIO  reconsider  its  response 
and  provide  additional  comments  in  response  to  the  final  report. 
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Appendix  A.  Audit  Process 

Scope 


Work  Performed.  We  conducted  this  audit  from  October  2000  through 
July  2001,  and  reviewed  documentation  dated  May  1997  through  July  2001. 

We  performed  audit  work  to  determine  the  adequacy  of  the  WAWF-RA 
application  development  and  implementation.  In  order  to  assess  development 
and  implementation,  we  used  the  Clinger-Cohen  Act  of  1996;  Office  of 
Management  and  Budget  Circular  A- 130,  “Management  of  Federal  Information 
Resources,”  February  8,  1996;  DoD  5000. 2-R,  “Mandatory  Procedures  for 
Major  Defense  Acquisition  Programs  (MDAPS)  and  Major  Automated 
Information  System  (MAIS)  Acquisition  Programs,”  October  23,  2000;  DoD 
Directive  8190.2,  “The  Department  of  Defense  (DoD)  Electronic 
Business/Electronic  Conamerce  (EB/EC)  Program,”  June  23,  2000;  and  the 
Defense  Reform  Initiative  Directives  Nos.  33  and  43. 

In  accomplishing  the  objective,  we  reviewed  how  DoD  and  DEBPO  managed 
the  development  of  WAWF-RA  and  the  extent  of  implementation  by  the 
Military  Departments.  We  reviewed  program  documentation  and  interviewed 
individuals  who  are  involved  in  the  development  and  implementation  of 
WAWF-RA.  Specifically  we: 

•  reviewed  WAWF-RA  working  group  minutes  from  August  1998  through 
March  2000, 

•  reviewed  the  results  of  a  working  group  review  that  developed 
recommendations  to  streamline  the  material  inspection  and  receiving 
report  process  with  the  aim  of  making  it  paperless, 

•  reviewed  WAWF-RA  draft  implementation  plans  provided  by  the 
services, 

•  reviewed  contract  documentation  from  July  1998  through  January  2001, 

•  interviewed  WAWF-RA  representatives  from  the  Army,  Navy,  Air 
Force,  DFAS,  and  Defense  Contract  Management  Agency, 

•  interviewed  DoD  and  Military  Department  Chief  Information  Officers, 
and 

•  visited  DoD  and  contractor  end-users  of  WAWF-RA  located  at 
DFAS-Columbus,  DCMA-Raytheon  in  Tucson,  Arizona,  DFAS-Pacific, 
and  Hickam  Air  Force  Base. 
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General  Accounting  Office  High-Risk  Area.  The  General  Accounting  Office 
has  identified  several  high-risk  areas  in  the  DoD.  This  report  provides  coverage 
of  the  DoD  Systems  Modernization  high-risk  area. 

Methodology 


Use  of  Computer-Processed  Data.  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Use  of  Technical  Assistance.  We  did  not  use  technical  assistance  to  perform 
this  audit. 

Audit  Type,  Dates,  and  Standards.  We  performed  this  economy  and 
efficiency  audit  from  October  2000  through  July  2001  in  accordance  with 
generally  accepted  government  auditing  standards. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  DoD.  Further  details  are  available  on  request. 

Management  Control  Program  Review 


DoD  Directive  5010.38,  “Management  Control  (MC)  Program,” 

August  26,  1996,  and  DoD  Instruction  5010.40,  “Management  Control  (MC) 
Program  Procedures,”  August  28,  1996,  require  DoD  organizations  to 
implement  a  comprehensive  system  of  management  controls  that  provides 
reasonable  assurance  programs  are  operating  as  intended  and  to  evaluate  the 
adequacy  of  the  controls. 

Scope  of  the  Review  of  the  Management  Control  Program.  We  reviewed  the 
adequacy  of  management  controls  over  the  development  of  new  technology.  We 
reviewed  management’s  self-evaluation  applicable  to  those  controls. 

Adequacy  of  Management  Controls.  We  identified  a  material  management 
control  weakness  for  DEBPO  as  defined  by  DoD  Instruction  5010.40.  The 
DEBPO  could  not  provide  reasonable  assurance  of  the  adequacy  of  controls  over 
the  development  of  new  technology.  Recommendation  2.f.,  if  implemented, 
will  improve  the  controls  over  the  development  of  new  technology.  A  copy  of 
this  report  will  be  provided  to  the  senior  official  responsible  for  management 
controls  at  DEBPO. 

Adequacy  of  Management’s  Self-Evaluation.  DEBPO  does  not  have  an 
independent  Management  Control  Program.  DEBPO  is  a  component  of  both  the 
Defense  Logistics  Agency  and  the  Defense  Information  Systems  Agency  and 
should  be  included  in  the  Defense  Logistics  Agency  and  Defense  Information 
Systems  Agency  Management  Control  Programs.  However,  neither  the  Defense 
Logistics  Agency  nor  the  Defense  Information  Systems  Agency  officials 
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identified  the  development  of  new  technology  as  an  assessable  unit.  Therefore, 
the  officials  did  not  identify  or  report  the  material  management  control  weakness 
identified  by  the  audit. 

Prior  Coverage 


General  Accounting  Office 

GAO  Report  No.  GAO-01-244,  “Major  Management  Challenges  and  Program 
Risks:  Department  of  Defense,”  January  1,  2001 

GAO  Report  No.  NSIAD-00-108,  “Electronic  Commerce  Implementation 
Strategy  Can  Be  Improved,”  July  18,  2000 
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Appendix  B.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Assistant  Secretary  of  Defense  for  Command,  Control,  Communications,  and 
Intelligence 

Director,  E-Business  and  Knowledge  Management 
Director,  Defense  Electronic  Business  Program  Office 


Department  of  the  Army 

Chief  Information  Officer 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Chief  Information  Officer 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 

Chief  Information  Officer 

Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Contract  Management  Agency 
Director,  Defense  Finance  and  Accounting  Service 
Director,  Defense  Information  Systems  Agency 
Director,  Defense  Logistics  Agency 


Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 
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Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcoirunittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Reform 

House  Subcommittee  on  Government  Efficiency,  Financial  Management,  and 
Intergovermnental  Relations,  Committee  on  Government 
House  Subcommittee  on  National  Security,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subcommittee  on  Technology  and  Procurement  Policy,  Committee  on 
Government  Reform 
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Assistant  Secretary  of  Defense  (Command, 
Control,  Communications,  and  Intelligence) 
Comments 


CHIEF  INFORMATION  OFFICER 


DEPARTMENT  OF  DEFENSE 

6000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-6000 


October  10,  2001 


MEMORANDUM  FOR  INSPECTOR  GENERAL,  DOD 

SUBJECT:  Response  to  DraR  Audit  Report  on  Development  and  Implementation  of  Wide 
Area  Workflow-Receipts  and  Acceptances  (WAWF-RA) 

Thank  you  for  the  opportunity  to  provide  comments  on  your  Draft  Audit  Report  on 
WAWF-RA.  We  have  reviewed  the  draft  and  the  attached  provides  DoD  CIO  specific 
comments  on  the  report.  This  ofdce  concurs  with  the  six  recommendations  and  has 
provided  amplifying  information  in  the  attachment. 

The  DoD  CIO  point  of  contact  is  Ms.  Dee  Ann  Sullivan,  (703)  604-1585, 


Paul  D.  Grant 

Director,  Electronic  Business  and  Knowledge  Management 


Attachment: 
As  Stated 
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Management  comments  follow  in  response  to  findings  and  applicable 
recommendations. 

Finding:  DEBPO  did  not  analyze  and  revise  the  receipts  and  acceptance 
before  it  began  developing  the  WAWF-RA. 

Recommendation  (a):  Determine  and  document  whether  the  material 
inspection  and  receiving  report  process  was  re-engineered  and  not  just  made 
paperless!^ 

CONCUR  WITH  COMMENTS:  Re-engineering  of  the  material 
inspection  and  receiving  report  process  was  recommended  imder  DRID  33 
and  incoiporated  into  DRID47,  creating  a  "To  Be”  procurement  process  and 
system  model.  The  model  looks  to  WAWF-RA  to  provide  the  mechanism 
for  re-engineering  the  material  inspection  and  receiving  process.  This 
includes  on-line  creation  of  documents,  with  up  firont  data  validation, 
system  to  system  sharing  of  data  eliminating  the  need  to  manually  re-key 
data  and  the  errors  associated  with  that  manual  process,  and  the  sharing  of 
data  electronically  between  vendors  and  various  Government 
representatives  involved  in  the  process. 

With  each  version  release  of  WAWF-RA,  all  requirements  targeted  for  a 
particular  release  are  discussed  by  the  WAWF-]^  Joint  Requirements 
Board,  allowing  for  an  ongoing  re-engineering  of  the  process  as  new 
requirements  and  functionality  are  included  and  new  versions  of  the 
software  released.  As  an  example,  version  2.0a  took  full  advantage  of 
changes  in  the  FAR  allowing  the  system  to  move  away  firom  the  traditional 
forms-based  process,  with  many  different  forms  being  used  for  the  same 
process,  to  a  single  standard  format  capturing  all  of  the  data  required  for 
any  of  the  paper-based  forms. 

The  initial  goal  to  building  WAWF-RA  was  to  make  the  receipts  and 
acceptance  process  paperless.  WAWF-RA  has  achieved  that  goal. 
Development  of  WAW-RA  was  preceded  by  the  Paperless  Contracting 
AS-IS  modeling  efforts,  the  follow-on  Procurement-Finance  TO-BE 
modeling,  and  the  DD250  Redesign  that  the  Defense  Contract  Management 
Agency  (DCMA)  conducted.  DEBPO  participated  in  each  of  these  forums 
and  tried  to  incorporate  the  best  “re-engineered”  ideas  where  there  was 
agreement.  These  efforts  established  a  formdation  upon  which  to  build 
WAWF-RA.  WAWF-RA  is  now  an  e-commerce  application  that  is 
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operating  in  a  production  mode.  It  is  available  for  all  Services/Agencies  to 
use. 

Finding:  DEBPO  did  not  develop  a  WAWF-RA  acquisition  strategy. 

Recommendation  (b):  Develop  an  acquisition  strategy  that  defines 
quantifiable  requirements,  timeframes  and  baselines,  and  measures  of 
performance  for  the  paperless  receipts  and  acceptance  process; 

CONCUR  WITH  COMENTS:  DEBPO  was  originally  set  up  to 
"improve  DoD  acquisition  processes  and  supporting  sustainment  life  cycle 
practices"  (PBD71 IR).  PBD085  provided  imds  to  DISA  to  "address  the 
unmatched  disbursements  problem  and  paperless  contracting  objectives." 
So  WAWF-RA  was  an  essential  part  of  the  directions  that  DISA  and  DLA 
received  when  DEBPO  was  being  stood  up.  Without  WAWF-RA,  DEBPO 
could  not  address  the  unmatched  disbursement  problem  or  fulfill  the 
paperless  contracting  objection  of  the  DoD.  The  DEBPO  charter  was  to 
“jump  start”  electronic  commerce  rather  than  to  create  formal  programs 
under  DoD  5000  rules.  After  initial  deployment,  they  were  to  be  turned 
over  to  Executive  Agents,  who  would  then  instantiate  them  into  formal 
programs. 

Although  there  is  no  formal  WAWF-RA  acquisition  plan  or  strategy,  a 
process  has  been  implemented  that  defines  quantifiable  requirements, 
timefinmes  and  baselines  through  the  auspices  of  the  Joint  Requirements 
Board  for  Wide-Area  Workflow.  Further,  measures  for  paperless  receipts 
and  acceptance  process  have  been  included  in  the  metrics  tracked  by  the 
Paperless  Contracting  Working  Integrated  Process  Team  and  continue  to  be 
tracked  as  that  effort  transferred  to  the  DoD  Cross-Functional 
Implementation  Integrated  Process  Team.  DEBPO  will  formally 
document  the  WAWF-RA  acquisition  plan  by  31  Dec  01. 

Finding:  DEBPO  did  not  prepare  a  memorandum  of  agreement  (MOA) 
with  DOD  Components. 

Recommendation  (c):  Develop  a  memorandum  of  agreement  with  the 
Military  Departments  and  Defense  Agencies  which  specifically  outlines  and 
details  the  relationship  and  responsibilities  for  the  Wide  Area  Work  Flow  - 
Receipts  and  Acceptance. 

CONCUR:  Responsibility  for  developing  MOAs  rests  with  DEBPO  until 
WAWF-RA  is  turned  over  to  the  Executive  Agents.  DEBPO  will 
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complete  development  of  the  MOAs  in  coordination  with  the  Executive 
Agents  by  30  Jun  02. 

Finding:  DEBPO  did  not  conduct  acquisition  decision  analysis. 

Recommendation  (d):  Perform  and  document  an  analysis  of  alternatives 
which  identifies  the  advantages  and  disadvantages  of  current  technology  and 
the  current  version  of  Wide-Area  Workflow-Receipts  and  Acceptance; 

PARTIALLY-CONCUR:  Initially  a  proof  of  concept  examined 
alternatives  through  the  demonstration  of  four  independent  technology 
solutions  in  1998,  followed  by  a  comprehensive  search  of  COTS  products. 
This  analysis,  although  not  documented,  led  us  to  the  technology  selected, 
which  was  a  proven  product  in  other  initiatives.  Subsequently,  Wide-Area 
Woricflow-Receipts  and  Acceptance  (WAWF-RA)  has  deployed  version 
2.0.a,  adding  additional  functionality  to  the  prototype  version  1.3.b.  A  new 
analysis  at  this  time  comparing  WAWF-RA  to  current  technologies  will  not 
impact  the  acquisition  or  the  cost  to  the  Government  in  any  significant  way, 
as  the  effort  is  complete.  This  analysis  was  conducted  on  September  1998^ 
As  plans  for  future  WAWF-RA  versions,  incorporating  additional 
functionality  are  developed,  alternative  technologies  will  be  evaluated  and 
documented  through  the  WAWF-RA  Joint  Requirements  Board  as  the  IG 
has  recommended.  In  addition,  COTS  solutions  for  workflow  functionality 
are  fi*equently  being  evaluated  by  the  DEBPO. 

Recommendation  (e):  Perform  and  document  an  economic  analysis  that 
includes  the  total  cost  and  benefit  for  each  feasible  alternative; 

CONCUR  with  Comments:  WAWF-RA  started  as  a  prototype, 
demonstrating  the  potential  of  an  electronic  receipt  and  acceptance  system. 
The  costs  of  die  development  and  testing  of  the  current  version  are  sunk 
costs  and  an  analysis  of  comparing  alternatives  will  not  impact  that 
expenditure.  As  the  system  moved  fix)m  prototyping  to  production,  the 
Joint  Requirement  Board  for  WAWF-RA  was  established.  As  new 
requirements  are  reviewed,  the  process  now  includes  an  evaluation  of 
alternate  technologies  and  the  associated  cost  and  benefits  for  different 
approaches.  We  concur  that  a  formal  economic  analysis  was  not 
documented  for  WAWF-RA.  Although  evaluating  the  costs  for  the  current 
version  of  WAWF-RA  to  other  technologies  at  this  time  would  seem  to  be 
of  somewhat  limited  value,  DEBPO  will  perform  and  document  an 
economic  analysis  to  be  used  as  a  baseline  in  evaluating  technologies  for 
future  releases.  This  economic  analysis  will  be  completed  by  30  Jun  2002. 
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Finding:  DEBPO  did  not  record,  preserve,  and  make  available  sufficient 
information  to  substantiate  the  acquisition  and  development  of  WAWF-RA. 

Recommendation  (f):  Develop  an  independent  Management  Control 
Program  that  provide  reasonable  assurance  with  clear  control  objectives 
that  the  development  of  new  information  technology  complies  with  the 
requirements  of  the  Clinger*-Cohen  Act  of  1996  and  the  Office  of 
Management  and  Budget  Circular  A-130. 

CONCUR:  Because  DEBPO  was  an  evolving  program  management 
organization,  a  true  software  engineering  infrastructure  was  not  in  place  at 
the  DEBPO.  Early  on,  there  was  no  c^trally  controlled  system  to  manage 
contractor  deliverables.  Individuals  maintained  configuration  control  on 
their  own  PCs.  However,  about  a  year  ago,  DEBPO  put  in  place  a 
configuration  management  system  called  PVCS.  All  WAWF-RA 
contractor  documents  and  software  deliveries  are  properly  managed  and 
tracked  in  this  system.  In  addition,  the  OASD  Deputy  Chief  Information 
Officer  established  a  Senior  Steering  Group  and  a  Joint  Requirements 
Board  for  the  Wide  Area  Workflow  Program  on  15  May  2000.  This  Joint 
Requirements  Board  defines  and  prioritizes  functional  requirements, 
recommends  system  enhancements,  oversees  functional  and  operational 
testing,  and  facilitates  cross-agency  planning,  training  and  deployment.  To 
further  improve  oversight  by  the  project  manager,  a  monthly  structured 
project  review  process  has  been  refined  to  carefully  scrutinize  WAWF-RA 
cost,  schedule  and  performance.  Taken  together,  these  initiative  clearly 
place  the  WAWF-RA  leadership  in  a  position  to  assure  management 
commitment  and  oversight  leac^g  to  a  successful  implementation  of 
WAWF-RA.. 


Miscellaneous  Comments: 

1 .  In  the  section  “Policy  and  Directives”  we  request  that  the  documents  listed 
below  be  included,  (a)  DRID  No.47  superceded  DRID  No.  33,  (b)  the  Sep  1, 2000 
Memorandum  establishing  a  process  for  implementing  DRID  No.  47,  and  (c)  May 
1 5, 2001  memorandum  for  integrating  WAWF-RA  into  the  DRID  No.  47 
environment. 

a.  Defense  Reform  Initiative  Directive  no.  47.  The  Deputy  Secretary  of 
Defense  published  DRID  No.  47,  dated  9  Dec  98,  directing  the 
establishment  of  a  DoD-wide  Working  Integrated  Process  Team  (WIPT)  to 
develop  and  document  the  future  end-to-end  procurement  process, 
incorporating  the  findings  and  recommendations  of  DRID  32.  The  product 
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of  the  WIPT  was  a  “To-Be”  procurement  process  model  and  system  model 
and  a  Strategic  Implementation  Plan.  Bo^  the  system  model  and  Strategic 
Implementation  Plan  identify  Wide  Area  Workflow  as  a  key  component  for 
achieving  the  To-Be  electronic  procurement  process. 

b*  Memorandum  dated  Sep  1, 2000  establishing  a  DoD-wide  Integrated 
Process  Team  for  the  implementation  to  facilitate  the  implementation 
DRID  #47  End-to-End  Procurement  Process  Model. 

c.  Management  Structure  for  DoD  Wide-Area  Workflow  Program. 

The  Deputy  Giief  Information  Officer  for  DoD  published  a  memorandum 
on  May  15, 2001  directing  the  establishment  of  a  Senior  Steering  Group 
and  a  Joint  Requirements  Board  for  the  Wide-Area  Workflow  program. 

2.  In  describing  WAWF-RA  Acceptance  and  Use  no  mention  was  made  of  the  six 
DCMA  pilot  sites.  We  would  recommend  including  language  similar  to  the 
following: 

“In  addition  to  the  Military  Department’s  pilot  sites  DCMA  has  six  pilot 
sites  and  is  in  the  process  of  establishing  four  additional  pilot  sites  with 
version  2.0.b.” 

3.  In  describing  the  Analysis  of  Alternatives,  the  DODIG  report  lists  four  systems, 
in  several  stages  of  development  that  made  the  process  paperless,  (a)  Defense 
Contract  Management  Command  Electronic  Data  Interchange  DD  Form  250,  (b) 
Paperless  Contracting  Working  Integrated  Process  Team  Wide  Area  Workflow, 

(c)  Federal  Systems  Integration  and  Management  Center  Tracking  and  Ordering 
System,  and  (d)  Defense  Medical  Logistics  Standard  Support.  We  recommend 
that  the  listing  of  these  systems  be  removed  from  the  report  for  the  following 
reasons. 

a.  Defense  Contract  Management  Command  Electronic  Daita  Interchange  DD 
Form  250  is  not  a  system.  DCMC  is  a  DOD  Agency.  Electronic  Data 
Interchange  is  an  American  National  Standards  Institute  (ANSI)  Accredited 
Standards  Committee  (ASC)  X.i2  Transaction  Set  for  electronic 
interchange.  WAWF-RA,  as  well  as  m^y  other  DEBPO  applications, 
utilize  the  EDI  standard  in  their  applications  to  electronically  transfer  data 
in  a  standard  format. 

b.  Paperless  Contracting  Working  Integrated  Process  Team  Wide  Area 
Workflow  is  not  a  system.  It  was  a  working  group  of  Service/Agency 
representatives  who  participated  in  the  evolution  of  WAWF-RA.  The 
group  provided  Service/Agency  requirements  to  the  DEBPO  and  the 


5 


28 


WAWF-RA  development  team.  The  working  group  was  transitiemed  to  the 
WAWF-RA  Joint  Requirements  Board. 

c.  Federal  Systems  Integration  wd  Management  Center  Tiacking  and 
Ordering  System.  This  system  has  no  rdatiemship  to  WAWF-RA. 

d.  Defense  Medical  Lo^stics  Standard  Suj^ort  (DMLSS).  This  system  has 
no  rclsdGMhijp  to  WAWF-RA  and  does  not  use  WAWF-RA  in  its  process. 
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The  Finance  and  Accounting  Directorate,  Office  of  the  Assistant  Inspector  General  for 
Auditing,  DoD,  prepared  this  report.  Personnel  of  the  Office  of  the  Inspector  General, 
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